Warning device, warning method, and warning program

ABSTRACT

To issue an appropriate warning based on detection of an object even under the circumstances where it is difficult to determine the outside environment of a movable body, a warning device according to the present invention includes an image acquisition unit configured to acquire a plurality of images respectively based on a plurality of filter characteristics, a detection unit configured to perform detection of a specified object on each of the plurality of acquired images, and a warning unit configured to issue a specific warning when the object is detected from at least any one of the plurality of acquired images, wherein the warning unit issues a higher level of warning when the object is detected from all of the plurality of images than when the object is detected from some of the plurality of images.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of PCT/JP2016/001136 filed on Mar. 2,2016, which is based upon and claims the benefit of priority fromJapanese patent application No. 2015-046584 filed on Mar. 10, 2015 andJapanese patent application No. 2015-046585 filed on Mar. 10, 2015, thedisclosures of which are incorporated herein in their entirety byreference.

BACKGROUND

The present invention relates to a warning device, a warning method, anda warning program.

An imaging device such as a digital camera generally takes an image ofan area to be imaged by using a filter that transmits light with aspecific wavelength. The imaging device can improve the image quality ofan image taken by use of a filter that is suitable for the amount oflight or the like of an area to be imaged. Thus, in the daytime with alarge amount of visible light, a camera is used that employs a filterfor daytime which transmits more light in the visible spectrum, and inthe nighttime with a small amount of visible light, a camera is usedthat employs a filter for nighttime which transmits more light in theinfrared spectrum.

Japanese Unexamined Patent Application Publication No. H07-160957discloses a technique related to a monitoring device that includes aninfrared sensor and a near-infrared camera. The monitoring devicedisclosed in Japanese Unexamined Patent Application Publication No.H07-160957 takes an image of an area monitored by the infrared sensor byusing the near-infrared camera, and detects a change in the monitoredarea based on the image taken by the near-infrared camera. Themonitoring device then determines an abnormal state based on thedetected change and the output of the infrared sensor.

Recently, a warning device has been developed that takes an image of thearea around a vehicle by using an in-vehicle camera, performs imagerecognition of the taken image and detects a person, a vehicle or thelike as an object in the image, and issues a warning to a driver.Japanese Unexamined Patent Application Publication No. 2011-146049discloses a technique related to a human detection device for detectinga person from a taken image. The human detection device disclosed inJapanese Unexamined Patent Application Publication No. 2011-146049detects the appearance of an object based on an image acquired from animage sensor, determines whether a parameter related to the detectedappearance of the object and a comparative parameter related to a humanmatch or not, calculates a human probability indicating the probabilitythat the object is a human according to the degree of matching, anddetermines that the object is a human when the human probability isequal to or more than a preset human threshold.

SUMMARY

In the case of the warning device described above, there arecircumstances where it is difficult to determine the outside environmentof a vehicle. The circumstances where it is difficult to determine theoutside environment of a vehicle include the circumstances where weatherconditions vary, the circumstances where a vehicle travels through aplace where the surrounding brightness suddenly changes before and afterentering a tunnel and the like. Therefore, there is a problem that it issometimes difficult to issue a warning based on detection of an objectunder the circumstances where it is difficult to determine the outsideenvironment of a vehicle.

Further, in the warning device described above, when an object exists inthe traveling direction of a vehicle, it is necessary to issue a warningto a driver because there is a possibility of a collision. Thepossibility of a collision can be calculated by predicting the relativemoving speed, the moving direction or the like of an object fromanalysis results of images taken in succession while a vehicle istraveling. However, the accuracy of detecting the movement of an objectcan decrease under the circumstances where it is difficult to determinethe outside environment of a vehicle, and there is a problem that it issometimes difficult to issue an appropriate warning.

Note that, in the monitoring device disclosed in Japanese UnexaminedPatent Application Publication No. H07-160957, an area to be monitoredis a specific area in facility, and it does not assume the circumstanceswhere it is difficult to determine the outside environment. Further, inthe human detection device disclosed in Japanese Unexamined PatentApplication Publication No. 2011-146049, a target to be analyzed is asingle image. Therefore, in the case where an appropriate filter cannotbe used due to the circumstances where it is difficult to determine theoutside environment, for example, the image quality is degraded. Whenthe image quality of an image to be analyzed is low, the accuracy ofdetecting an object that can interfere with the traveling of a vehiclecan decrease. This causes a failure to issue an appropriate warning.Further, such a problem occurs in a warning device that is mounted onmovable bodies in general, not limited to vehicles.

A first aspect of the embodiment provides a warning device that includesan image acquisition unit configured to acquire a plurality of imagesrespectively based on a plurality of filter characteristics, a detectionunit configured to perform detection of a specified object on each ofthe plurality of acquired images, and a warning unit configured to issuea specific warning when the object is detected from at least any one ofthe plurality of acquired images, wherein the warning unit issues ahigher level of warning when the object is detected from all of theplurality of images than when the object is detected from some of theplurality of images.

A second aspect of the embodiment provides a warning method thatincludes an image acquisition step of acquiring a plurality of imagesrespectively based on a plurality of filter characteristics, a detectionstep of performing detection of a specified object on each of theplurality of acquired images, and a warning step of issuing a specificwarning when the object is detected from at least any one of theplurality of acquired images, wherein the warning step issues a higherlevel of warning when the object is detected from all of the pluralityof images than when the object is detected from some of the plurality ofimages.

A third aspect of the embodiment provides a non-transitory computerreadable medium storing a warning program that causes a computer toexecute an image acquisition step of acquiring a plurality of imagesrespectively based on a plurality of filter characteristics, a detectionstep of performing detection of a specified object on each of theplurality of acquired images, and a warning step of issuing a specificwarning when the object is detected from at least any one of theplurality of acquired images, wherein the warning step issues a higherlevel of warning when the object is detected from all of the pluralityof images than when the object is detected from some of the plurality ofimages.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing the overall configuration of a movablebody on which a warning device according to a first embodiment ismounted.

FIG. 2 is a block diagram showing the hardware configuration of thewarning device according to the first embodiment.

FIG. 3 is a flowchart illustrating the flow of an object detection andwarning process according to the first embodiment.

FIG. 4 is a view showing an example of an image taken by a first filteraccording to the first embodiment.

FIG. 5 is a view showing an example of an image taken by a second filteraccording to the first embodiment.

FIG. 6 is a flowchart illustrating the flow of an example of a warningprocess according to the first embodiment.

FIG. 7 is a view showing an example of displaying an image with theposition of an object highlighted according to the first embodiment.

FIG. 8 is a block diagram showing the configuration of a warning deviceaccording to a second embodiment.

FIG. 9 is a flowchart illustrating the flow of a mode setting processaccording to the second embodiment.

FIG. 10 is a block diagram showing the configuration of an imageacquisition means according to a third embodiment.

FIG. 11 is a block diagram showing the overall configuration of amovable body on which a warning device according to a fourth embodimentis mounted.

FIG. 12 is a block diagram showing the hardware configuration of thewarning device according to the fourth embodiment.

FIG. 13 is a flowchart illustrating the flow of a process from taking animage to issuing a warning according to the fourth embodiment.

FIG. 14 is a flowchart illustrating the flow of a detailed process in anexample 1 of the fourth embodiment.

FIG. 15 is a view illustrating the concept of processing in the example1 of the fourth embodiment.

FIG. 16 is a view showing an example of displaying an image with theposition of an object highlighted in the example 1 of the fourthembodiment.

FIG. 17 is a flowchart illustrating the flow of a detailed process in anexample 2 of the fourth embodiment.

FIG. 18 is a view illustrating the concept of processing in the example2 of the fourth embodiment.

FIG. 19 is a flowchart illustrating the flow of a detailed process in anexample 3 of the fourth embodiment.

FIG. 20 is a view illustrating the concept of processing in the example3 of the fourth embodiment.

DETAILED DESCRIPTION

Specific exemplary embodiments of the present invention will bedescribed hereinafter in detail with reference to the drawings. It isnoted that in the description of the drawings, the same elements will bedenoted by the same reference symbols and redundant description will beomitted to clarify the explanation.

First Embodiment of the Invention

FIG. 1 is a block diagram showing the overall configuration of a movablebody 1 on which a warning device 10 according to a first embodiment ismounted. The movable body 1 is equipped with the warning device 10, adisplay device 21, a speaker 22 and the like, and travels by operationof a driver. The movable body 1 is a vehicle such as an automobile, forexample. The warning device 10 is a device that analyzes an image thatis taken while the movable body 1 is traveling and thereby detects anearby object that can collide with the movable body 1, and issues awarning to a driver through the display device 21, the speaker 22 or thelike. Note that the display device 21 is an organic EL(electro-luminescence) display or a plasma display, for example.

The warning device 10 includes an image acquisition means 11, adetection means 12, and a warning means 13. The image acquisition means11 takes images of the area around the movable body 1 by a plurality ofdifferent filter characteristics while the movable body 1 is traveling,and acquires them as a plurality of images. In other words, the imageacquisition means 11 acquires a plurality of images based on theplurality of filter characteristics, respectively. The image acquisitionmeans 11 includes imaging devices 111 and 112. The imaging devices 111and 112 respectively correspond to the plurality of filtercharacteristics. Each of the imaging devices 111 and 112 takes images bythe corresponding filter characteristics. Note that the number ofimaging devices included in the image acquisition means 11 is notlimited to two. For example, the image acquisition means 11 may includethree or more imaging devices respectively corresponding to three ormore filter characteristics. Note that the imaging device is a camera,for example.

The detection means 12 performs detection of a specified object on eachof the plurality of acquired images. The detection of an object is theprocessing of trying to detect the appearance of an object by imagerecognition.

For example, there is a case where a person who is crossing a road isshown, together with the shape of the road, in an image taken by acamera in the traveling direction of a vehicle. In such a case, the areashowing the appearance of the person who is crossing the road can bedetected. The detection means 12 generates the detection result as anindex value. Further, as described in Japanese Unexamined PatentApplication Publication No. 2011-146049, the detection means 12 maydetect a parameter related to the appearance of an object and generatethe degree of matching between a comparative parameter and the detectedparameter as a detection result. Note that the processing of generatinga detection result when detecting an object from each image is notlimited thereto. Further, the object is not limited to a human, and itincludes any movable body that can interfere with the traveling of themovable body 1, such as an animal and another vehicle. Furthermore, theobject is not necessarily a movable body, and it may be a personstanding on the road, a stopped vehicle, a road closed sign, an obstacleleft on the road and the like. Note that the parameter related to theappearance may be luminance, saturation, edge, or a combination ofthose.

The warning means 13 issues a specific warning when an object isdetected from at least any one of the plurality of acquired images.Particularly, the warning means 13 according to an embodiment of thepresent invention issues a higher level of warning when an object isdetected from all of the plurality of images than when an object isdetected from only some of the plurality of images. Issuing a higherlevel of warning means changing the operation of a warning means so thatan object is more clearly recognizable, as described later.

FIG. 2 is a block diagram showing the hardware configuration of thewarning device 10 according to the first embodiment. The warning device10 includes a processor 110, an IF (InterFace) unit 120, imaging devices131 and 132, and a storage unit 140.

The processor 110 is a control device such as a CPU (Central ProcessingUnit). The IF unit 120 is an interface for inputting and outputting datato and from the outside of the warning device 10. Particularly, the IFunit 120 outputs information indicating a warning to the display device21 or the speaker 22 under control of the processor 110.

The imaging devices 131 and 132 respectively correspond to the imagingdevices 111 and 112 described above. The imaging device 131 includes afirst filter 1311 and a first image sensor 1312. The first filter 1311makes selection of the wavelength of light to be incident on the firstimage sensor 1312. Thus, the first filter 1311 is a filter that isapplied when taking an image by the imaging device 131. The first filter1311 has the filter characteristics that block light in thenear-infrared spectrum and transmit light in the visible spectrum, forexample. It is assumed that the first filter 1311 is at least a filterthat is suitable in the case where the area around the movable body 1 isbright such as during daytime hours. The first image sensor 1312 is anelement that converts the intensity of light that has passed through thefirst filter 1311 into an electrical signal.

The imaging device 132 includes a second filter 1321 and a second imagesensor 1322. The second filter 1321 makes selection of the wavelength oflight to be incident on the second image sensor 1322. Thus, the secondfilter 1321 is a filter that is applied when taking an image by theimaging device 132. The second filter 1321 has the filtercharacteristics that block light in the near-infrared spectrum andtransmit light in the visible spectrum, for example. It is assumed thatthe second filter 1321 is at least a filter that is suitable in the casewhere the area around the movable body 1 is dark such as duringnighttime hours. The second image sensor 1322 is an element thatconverts the intensity of light that has passed through the secondfilter 1321 into an electrical signal.

The storage unit 140 is a storage device such as a memory or a harddisk, for example. The storage unit 140 stores a warning program 141,taken images 1421 and 1422, and index values 1431 and 1432. The warningprogram 141 is a computer program in which object detection and warningprocessing, which is a warning method, of the warning device 10according to this embodiment is implemented.

The taken image 1421 is image data that is taken by the imaging device131 with use of the first filter 1311 and the first image sensor 1312.The taken image 1422 is image data that is taken by the imaging device132 with use of the second filter 1321 and the second image sensor 1322.The index value 1431 is a value indicating the level of detection of anobject from the taken image 1421 by the detection means 12. The indexvalue 1432 is a value indicating the level of detection of an objectfrom the taken image 1422 by the detection means 12. The index values1431 and 1432 are, in other words, index values of the accuracy ofdetecting an object when image recognition is performed on taken images.

The processor 110 reads the warning program 141 from the storage unit140 and executes it. The warning device 10 thereby operates as the imageacquisition means 11, the detection means 12, the warning means 13 andthe like according to this embodiment by using the IF unit 120, theimaging devices 131 and 132 as appropriate.

FIG. 3 is a flowchart illustrating the flow of the object detection andwarning process according to the first embodiment. It is assumed that itis in the early evening or the like, and the amount of light which isintermediate between the amount of light in the daytime and the amountof light in the nighttime is incident on the movable body 1.

First, the imaging device 111 takes an image of the area around themovable body 1 by the first filter characteristics while the movablebody 1 is traveling, and acquires it as a first image (S111). At thistime, the warning device 10 stores the acquired first image into thestorage unit 140. Then, the detection means 12 performs detection of aspecified object on the first image (S112). FIG. 4 is a view showing anexample of a taken image 30 a by the first filter according to the firstembodiment. It shows that, in the taken image 30 a, an object 41 isdetected to some degree in the traveling direction of the movable body1. However, the detection means 12 cannot sufficiently identify theobject 41.

In parallel with Step S111, the imaging device 112 takes an image of thearea around the movable body 1 by the second filter characteristicswhile the movable body 1 is traveling, and acquires it as a second image(S121). At this time, the warning device 10 stores the acquired secondimage into the storage unit 140. Then, the detection means 12 performsdetection of a specified object on the second image (S122). FIG. 5 is aview showing an example of a taken image 30 b by the second filteraccording to the first embodiment. It shows that, in the taken image 30b, an object 42 is slightly detected in the traveling direction of themovable body 1. However, because the taken image 30 b is blurred, thedetection means 12 cannot sufficiently distinguish the object 42 fromthe road.

After Steps S112 and S122, the warning means 13 determines whether anobject is detected from one of the first image and the second image(S131). When an object is detected from one image, the warning means 13performs first warning processing (S132). For example, the warning means13 outputs the image from which the object is detected to the displaydevice 21 and displays the image on the screen. For example, in the casewhere the amount of light in the outside environment is not sufficient,while an object is clearly shown in the first image, the object cannotbe recognized and detected in the second image. In such a case, thewarning means 13 outputs the first image to the display device 21. Withthe first image displayed on the screen, a driver can recognize thedanger.

When the determination in Step S131 results in No, the warning means 13determines whether an object is detected from both of the first imageand the second image (S133). For example, in the case of the takenimages 30 a and 30 b in FIGS. 4 and 5 described above, an object isdetected from both images. In this case, the warning means 13 performssecond warning processing (S134). The second warning processing issues ahigher level of warning than the first warning in Step S132. Theissuance of a higher level of warning is described hereinafter. Forexample, the warning means 13 outputs, to the display device 21, thefirst image in which the object is detected at higher accuracy among thefirst image and the second image, and displays it on the screen. At thistime, the warning means 13 may modify the first image to highlight thearea around the object in the first image and displays it on the displaydevice 21. Note that, as described later, highlighting may be done bydrawing a rectangle line around an object, placing a mark on or above anobject, changing the color of an object or the like, for example.Alternatively, the warning means 13 may output a warning tone throughthe speaker 22, together with display on the basis of the first image.In sum, the warning means 13 issues a specific warning by using one ofthe first image and the second image in which the object is detected athigher accuracy.

On the other hand, the case where the level of warning is not highcompared with the above (for example, when an object is detected fromone of the first image and the second image) is also described as anexample. In this case, one of the first image and the second image inwhich an object is shown is output to the display device 21 anddisplayed on the screen as described above. At this time, the warningmeans 13 performs processing such as displaying the first image on thedisplay device 21 without any modification, or, when modifying the imageand displaying it on the display device 21, performs processing such asdrawing a thinner line around an object, placing a different mark abovean object, or changing the color of an object to be modified comparedwith the case where the level of warning is high. Further, in the casewhere the warning means 13 outputs a warning tone through the speaker22, it performs processing such as changing the volume or changing thewarning tone compared with the case where the level of warning is high.

When the determination in Step S133 results in No, the process of FIG. 3ends.

In Steps S112 and S122 in FIG. 3, the detection means 12 may calculatean index value indicating the level of detection of an object for eachof the plurality of acquired images. For example, as described inJapanese Unexamined Patent Application Publication No. 2011-146049, thedetection means 12 may detect a parameter related to the appearance ofan object and calculate the degree of matching between a comparativeparameter and the detected parameter as the index value. The parameterrelated to the appearance may be luminance, saturation, edge, or acombination of those. Note that the processing of calculating the indexvalue when detecting an object from each image is not limited thereto.

Then, the detection means 12 stores the calculated index values into thestorage unit 140. An example of warning processing from Steps S131 toS134 in FIG. 3 in this case is described hereinafter with reference toFIG. 6.

First, the warning means 13 determines whether the index value exceeds aspecified value in one image or not (S131 a). For example, the warningmeans 13 determines whether the index value 1431 read from the storageunit 140 exceeds a specified value is one or not. Likewise, the warningmeans 13 determines whether the index value 1432 read from the storageunit 140 exceeds a specified value is one or not. When one of the indexvalues 1431 and 1432 exceeds a specified value and the other one of themfalls below the specified value, the warning means 13 displays thecorresponding image, which is the image in which the index value exceedsthe specified value, on the screen (S132 a). Step S132 a is one exampleof first warning processing.

When the determination in Step S131 a results in No, the warning means13 determines whether the index value exceeds a specified value in bothimages or not (S133 a). When the index value exceeds a specified valuein both images, the warning means 13 selects the image with the highestindex value (S134 a). For example, in the case of the taken images 30 aand 30 b, the taken image 30 a is selected. Then, the warning means 13displays, on the screen, the selected image with the object highlighted(S134 b). For example, the warning means 13 generates an image in whicha line is drawn around the area of the object 41 in the taken image 30a, outputs the image to the display device 21 and displays it on thescreen. FIG. 7 is a view showing an example of displaying an image withthe position of an object highlighted according to the first embodiment.A highlighted display area 43 in the taken image 30 a is one example ofhighlighting. Note that, however, highlighting is not limited thereto.When the determination in Step S133 a results in No, the process of FIG.6 ends.

As described above, the first embodiment has the following advantages.

First, in the case of taking an image by using any one of two filtercharacteristics as before, it is possible to detect an object with highaccuracy by use of appropriate filter characteristics for the outsideenvironment. However, the accuracy of detecting an object is degraded ifinappropriate filter characteristics are used. Further, when a movablebody is traveling, the circumstances where it is difficult to determinethe outside environment can occur. Therefore, in the first embodiment,images of the surrounding area are taken by using each of a plurality offilter characteristics, thereby increasing the probability of detectingan object from at least any one of the images. It is thus possible toissue a warning to a driver about a dangerous situation in more casesand thereby enhance the safety.

An example of the circumstances where it is difficult to determine theoutside environment is the early evening when it is difficult todistinguish between the daytime and the nighttime from the amount oflight. In such circumstances, each of the images taken by use of afilter for daytime and a filter for nighttime does not have sufficientquality, and there is a possibility that accurate detection cannot beachieved by analyzing a single image only. Thus, in the firstembodiment, it is determined as a dangerous situation when an object isdetected at a certain level of accuracy from each of a plurality ofimages obtained by taking the same object by a plurality of differentfilter characteristics. Particularly, in such circumstances, a driver islikely to have difficulty recognizing an object compared with the casewhere it is clearly daytime or nighttime. Therefore, the level ofwarning is increased in such circumstances, and it is thereby possibleto issue an appropriate warning to a driver.

Further, in the case of second warning processing in which the level ofwarning is higher than in first warning processing, one of two images inwhich an object is detected at higher accuracy is used. This is because,when an object is detected from two or more taken images, a warning isissued with use of the image with higher detection accuracy, so that adriver can more easily recognize the object.

Second Embodiment of the Invention

A second embodiment is a modified example of the first embodimentdescribed above. FIG. 8 is a block diagram showing the configuration ofa warning device 10 a according to the second embodiment. The warningdevice 10 a includes a position information acquisition means 14 and asetting means 15 in addition to the elements of the warning device 10shown in FIG. 1. Therefore, the same elements as those in FIG. 1 are notredundantly described below. Further, for the convenience ofdescription, a storage unit 16, which corresponds to the storage unit140 in FIG. 2, is shown in FIG. 8. It is assumed that functionscorresponding to those described in the second embodiment are alsoimplemented in the warning program 141.

The storage unit 16 further includes map information 161, specific area162, position information 163, first mode 164 and second mode 165, inaddition to the elements in the storage unit 140 of FIG. 2. The mapinformation 161 is information that defines, by coordinates, thepositions of roads on which a vehicle travels and facilities. Note thatthe map information 161 may be the one used in a general car navigationsystem. The specific area 162 is information that defines a partial areain the map information 161 in advance. The specific area 162 indicatesan area on an expressway, on a local street, inside a tunnel, near atrain station or the like. The position information 163 is the currentposition information of the movable body 1 that is acquired by theposition information acquisition means 14.

The first mode 164 is setting information that defines that the imageacquisition means 11 takes an image by one filter characteristics. Thesecond mode 165 is setting information that defines that the imageacquisition means 11 takes an image by a plurality of filtercharacteristics.

The position information acquisition means 14 acquires the currentposition information of the movable body 1 at regular intervals. Theposition information acquisition means 14 then stores the acquiredposition information into the storage unit 16. The position informationacquisition means 14 is a device that is mounted on a vehicle and canacquire GPS (Global Positioning System) information, for example.

The setting means 15 selects the first mode 164 or the second mode 165according to specified conditions, and sets the selected mode to theimage acquisition means 11. Further, when the acquired positioninformation 163 is within the range of the specific area 162 on the mapinformation 161, the setting means 15 selects the second mode 165 andsets it to the image acquisition means 11.

The specified conditions are conditions using some or all of positioninformation, time information, weather, road information, drivinginformation and the like, for example. An example of the conditionsusing position information is whether the distance between the currentposition and a specific position is within a specified value or not, orwhether it is within the range of a specific area or not. An example ofthe conditions using time information is conditions for determining aseason based on the present day of month, or conditions for determininga time period such as the daytime, the nighttime or the evening based onhours and minutes. To use the time information, it is necessary to addan element for acquiring information about the current time or the like.

An example of the conditions using the weather is the state of clouds,whether the weather is sunny, cloudy, rainy or the like. To use theweather, it is necessary to add an element for acquiring informationabout the weather or the like. An example of the conditions using roadinformation is conditions for determining whether a road type is anexpressway or a local street, the width of road, a road sign, a roadshape, a road surface condition or the like. To use the roadinformation, it is necessary to add an element for acquiring informationabout the road on which a vehicle is traveling or the like. An exampleof the conditions using driving information is conditions fordetermining whether vehicle speed, brake control information or the likeexceeds a specified value or not. To use the driving information, it isnecessary to add an element for acquiring speed or brake controlinformation from a vehicle or the like.

FIG. 9 is a flowchart illustrating the flow of a mode setting processaccording to the second embodiment. First, the position informationacquisition means 14 acquires the current position information on aregular basis (S21). Next, the setting means 15 determines whether theacquired position information is within the range of the specific areaor not (S22). When it is determined that the acquired positioninformation is within the range of the specific area, the setting means15 selects the second mode (S23). On the other hand, when it isdetermined that the acquired position information is outside the rangeof the specific area, the setting means 15 selects the first mode (S24).After that, the setting means 15 sets the selected mode to the imageacquisition means 11 (S25).

As described above, in the second embodiment, it is possible to changethe type of a filter to be used for taking an image according tosituations. Thus, when it is easy to determine the outside environment,it is possible to use the first mode to reduce the power consumption orthe like of the movable body 1. On the other hand, when it is difficultto determine the outside environment, it is possible to immediatelychange the mode to the second mode using a plurality of filtercharacteristics.

Third Embodiment of the Invention

A third embodiment is a modified example of the first embodimentdescribed above. While the image acquisition means 11 includes twoimaging devices in the first embodiment described above, an imageacquisition means 11 a according to the third embodiment includes oneimaging device.

FIG. 10 is a block diagram showing the configuration of the imageacquisition means 11 a according to the third embodiment. The imageacquisition means 11 a includes one imaging device 113. The imagingdevice 113 takes images by switching a plurality of filtercharacteristics on a regular basis. The imaging device 113 includes afilter 1131, an image sensor 1132 and a switch means 1133. The filter1131 makes selection of the wavelength of light to be incident on theimage sensor 1132. For example, the filter 1131 selects and transmitslight in the visible spectrum and light in the near-infrared spectrum tobe incident on the image sensor 1132. Note that the filter 1131 mayinclude an infrared cut filter (IR filter), a high-pass filter, aband-pass filter and the like. Accordingly, the filter 1131 canimplement certain filter characteristics selected from a plurality oftypes of filter characteristics. The image sensor 1132 is an elementthat converts the intensity of light that has passed through the filter1131 into an electrical signal. The switch means 1133 sets the selectedfilter characteristics to the filter 1131. The switch means 1133 thenswitches the selection of filter characteristics at regular intervals.Note that, it is assumed that the switch means 1133 switches the filtercharacteristics at a speed which is substantially equal to that when aplurality of imaging devices take images of the same object, at the sametime, by a plurality of filter characteristics. The switching and imagetaking are done on a regular basis by the switch means having a clock orreceiving a clock signal.

As described above, the same advantages as in the first embodimentdescribed above can be obtained also in the third embodiment.

Fourth Embodiment of the Invention

FIG. 11 is a block diagram showing the overall configuration of amovable body 1 b on which a warning device 10 b according to a fourthembodiment is mounted. The movable body 1 b is equipped with the warningdevice 10 b, a display device 21, a speaker 22 and the like, and travelsby operation of a driver. The movable body 1 b is a vehicle such as anautomobile, for example. The warning device 10 b is a device thatanalyzes an image that is taken while the movable body 1 b is travelingand thereby detects a nearby object that can collide with the movablebody 1 b, and issues a warning to a driver through the display device21, the speaker 22 or the like. Note that the display device 21 and thespeaker 22 are the same as those of FIG. 1.

The warning device 10 b includes an image acquisition means 11 b, adetection means 12 b, a determination means 17, and a warning means 13b. The image acquisition means 11 b takes images of the area around themovable body 1 b successively in parallel by a plurality of differentfilter characteristics while the movable body 1 b is traveling. Theimage acquisition means 11 b then acquires the plurality of taken imagesas a plurality of time-series image groups for each of the filtercharacteristics. Stated differently, the image acquisition means 11 bacquires an image group including a plurality of images taken insuccession. Then, the image acquisition means 11 b acquires a pluralityof image groups based on the plurality of filter characteristics,respectively. The image acquisition means 11 b includes imaging devices111 and 112. The imaging devices 111 and 112 respectively correspond tothe plurality of filter characteristics. Each of the imaging devices 111and 112 takes images by the corresponding filter characteristics. Notethat the number of imaging devices included in the image acquisitionmeans 11 b is not limited to two. For example, the image acquisitionmeans 11 b may include three or more imaging devices respectivelycorresponding to three or more filter characteristics. Note that theimaging device is a camera, for example.

The detection means 12 b performs detection of a specified object oneach of images in the plurality of image groups. The detection means 12b then generates a detection result for each image. Note that a methodof detecting an object in one image by the detection means 12 b is thesame as the method by the detection means 12 described above.

The determination means 17 compares object detection results in aplurality of image groups in chronological order, and determines thedegree of movement of an object. “Comparing detection results inchronological order” means comparing detection results in images takenat the times adjacent to each other. At this time, detection results maybe compared between images in each image group, or between imagesbelonging to different image groups. Alternatively, detection resultsmay be compared between images belonging to different image groups andtaken at the same time. Further, an average of detection results betweenimages belonging to different image groups and taken at a certain timemay be calculated, and an average between images taken at an adjacenttime may be compared.

Further, “the degree of movement” includes the degree indicating whetheran object is moving closer to or farther from the movable body 1 b,which is, the degree of approach or the speed of approach of an objectto the movable body 1 b, the speed of movement or the direction ofmovement of an object and the like, for example. Further, “determiningthe degree of movement of an object” includes determining whether thespeed of approach or the like is higher than a reference value, forexample. Alternatively, when an object moves crossing a road, theprobability of a collision between the object and the movable body 1 bmay be calculated from the moving direction and the moving speed, thetraveling direction and the traveling speed of the movable body 1 b, andwhether the probability of a collision exceeds a reference value or notmay be determined, for example.

The warning means 13 b issues a warning in accordance with thedetermined degree of movement. For example, the warning means 13 bissues a warning when an object is approaching the movable body 1 b fromthe degree of movement. Particularly, the warning means 13 b issues ahigher level of warning when the degree of approach exceeds a referencevalue. Further, the warning means 13 b issues a higher level of warningalso when the probability of a collision exceeds a reference value asdescribed above.

FIG. 12 is a block diagram showing the hardware configuration of thewarning device 10 b according to the fourth embodiment. The warningdevice 10 b includes a processor 110, an IF unit 120, imaging devices131 and 132, and a storage unit 140 b. The processor 110, the IF unit120, the imaging devices 131 and 132 are the same as those in FIG. 2,and detailed description thereof is omitted.

The storage unit 140 b is a storage device such as a memory. The storageunit 140 b stores a warning program 141 b, a first image group 1421 b, asecond image group 1422 b, a detection result 143 and an approach speed144. The warning program 141 b is a computer program in whichprocessing, which is a warning method, of the warning device 10 baccording to the fourth embodiment is implemented.

The first image group 1421 b is a time-series set of a plurality oftaken images 14211, 14212, . . . that are obtained by taking images ofthe area around the movable body 1 in succession by the first filtercharacteristics while the movable body 1 b is traveling. It is assumedthat the images 14211, 14212, . . . are taken successively in thisorder, for example. Further, the second image group 1422 b is atime-series set of a plurality of taken images 14221, 14222, . . . thatare obtained by taking images of the area around the movable body 1 insuccession by the second filter characteristics, in parallel with takingimages in the first image group 1421 b. It is assumed that the images14221, 14222, . . . are taken successively in this order, for example.In sum, the images 14211 and the 14221 are taken at the correspondingtime, and the images 14212 and the 14222 are taken at the correspondingtime.

The detection result 143 is a result of trying to detect an object ineach of the taken images. The detection result 143 exists for each takenimage. The approach speed 144 is one example of the degree of approachof an object to the movable body 1 b, and it is a relative moving speedof an object to the movable body 1 b. The approach speed 144 iscalculated from detection results in a plurality of images or the likein consideration of the time the images are taken.

The processor 110 reads the warning program 141 b from the storage unit140 b and executes it. The warning device 10 b thereby operates as theimage acquisition means 11 b, the detection means 12 b, thedetermination means 17, the warning means 13 b and the like according tothe fourth embodiment by using the IF unit 120, the imaging devices 131and 132 as appropriate.

FIG. 13 is a flowchart illustrating the flow of a process from taking animage to issuing a warning according to the fourth embodiment. First,the imaging device 111 takes images of the area around the movable body1 b in succession by the first filter characteristics while the movablebody 1 b is traveling (S41). The image acquisition means 11 b thenacquires the taken images as the first image group (S42). To bespecific, each time an image is taken, the image acquisition means 11 bstores the taken image in association with the first image group 1421 bin the storage unit 140 b. Then, the detection means 12 b performsdetection of a specified object on each image in the first image group(S43). To be specific, the detection means 12 b reads the taken image14211 or the like that is associated with the first image group 1421 bstored in the storage unit 140 b, performs detection for each image, andstores a detection result into the storage unit 140 b.

Further, in parallel with Step S41, the imaging device 112 takes imagesof the area around the movable body 1 b in succession by the secondfilter characteristics while the movable body 1 b is traveling (S44).The image acquisition means 11 b then acquires the taken images as thesecond image group (S45). To be specific, each time an image is taken,the image acquisition means 11 b stores the taken image in associationwith the second image group 1422 b in the storage unit 140 b. Then, thedetection means 12 b performs detection of a specified object on eachimage in the second image group (S46). To be specific, the detectionmeans 12 b reads the taken image 14221 or the like that is associatedwith the second image group 1422 b stored in the storage unit 140 b,performs detection for each image, and stores a detection result intothe storage unit 140 b.

Note that the detection means 12 b may perform the detection processingin Steps S43 and S46 each time images are taken by the imaging devices111 and 112.

After that, the determination means 17 compares detection results in thefirst image group and the second image group in chronological order, anddetermines the degree of movement of the object (S47). For example, thedetermination means 17 calculates the speed of approach of the object tothe movable body 1 b based on the detection results, and determines thedegree of movement of the object by using the speed of approach. Then,the warning means 13 b issues a warning in accordance with thedetermined degree of movement (S48).

As described above, in the fourth embodiment, while the movable body 1 bis traveling, images of the area around the movable body 1 b are takenin succession in parallel by a plurality of filter characteristics. Atthis time, the taken images are sorted, in chronological order, intodifferent image groups corresponding to different filtercharacteristics. In the images taken by a plurality of filtercharacteristics, it is likely that the quality of an image taken by atleast one of the filter characteristics in each time period attains thequality equivalent to an image taken by appropriate filtercharacteristics even under the circumstances where it is difficult todetermine the outside environment of the movable body 1 b. Therefore, bycomparing object detection results on each of the images inchronological order, it is possible to determine the degree of movementof an object. It is thereby possible to issue an appropriate warning toa driver in accordance with the degree of movement of an object. Thisenhances the safety.

Particularly, by determining the degree of movement with use of thespeed of approach, it is possible to make a determination about thedanger of a collision more directly, which improves the accuracy ofwarning as well.

Examples 1 to 3 that show the detailed processing of Steps S47 and S48in FIG. 13 described above are described hereinbelow.

Example 1

In the example 1, the speed of approach is calculated for each imagegroup, and a warning is issued based on the image group with a higherspeed of approach. Processing of the example 1 is described hereinafterwith reference to FIG. 14, and with reference also to FIG. 15 accordingto need. FIG. 14 is a flowchart illustrating the flow of a detailedprocess in the example 1 of the fourth embodiment. FIG. 15 is a viewillustrating the concept of processing in the example 1 of the fourthembodiment. It is assumed that taken images 311 to 313 belong to a firstimage group 31 and taken at the time t1 to t3, respectively. It is alsoassumed that taken images 321 to 323 belong to a second image group 32and taken at the time t1 to t3, respectively.

In FIG. 14, the determination means 17 first makes a comparison betweenimages in the first image group in chronological order and calculatesthe speed of approach (S51). In the example of FIG. 15, thedetermination means 17 compares detection results between the takenimage 311 and the taken image 312 that belong to the first image group31 and also compares detection results between the taken image 312 andthe taken image 313 that belong to the first image group 31, and therebycalculates the moving distance of an object. Then, the determinationmeans 17 calculates the speed of approach of the object in the firstimage group 31 based on the moving distance and a time interval betweenfrom the time t1 to the time t3.

Likewise, the determination means 17 makes a comparison between imagesin the second image group in chronological order and calculates thespeed of approach (S52). In the example of FIG. 15, the determinationmeans 17 compares detection results between the taken image 321 and thetaken image 322 that belong to the second image group 32, and alsocompares detection results between the taken image 322 and the takenimage 323 that belong to the second image group 32, and therebycalculates the moving distance of the object. Then, the determinationmeans 17 calculates the speed of approach of the object in the secondimage group 32 based on the moving distance and a time interval betweenfrom the time t1 to the time t3. Note that Steps S51 and S52 are notnecessarily performed in parallel, as long as the determination means 17calculates the speed of approach for each image group.

Next, the determination means 17 selects an image group with a higherspeed of approach among the plurality of image groups (S53).Specifically, the determination means 17 compares the speeds of approachcalculated in Steps S51 and S52 and determines in which of the firstimage group 31 and the second image group 32 the speed of approach ishigher and selects one. Then, the determination means 17 determines thedegree of movement of the object by using the speed of approach in theselected image group (S54). For example, the determination means 17determines whether the selected speed of approach is equal to or higherthan a specified reference value or not, and when it is equal to orhigher than the specified reference value, determines that the object isapproaching the movable body 1 b at a higher speed than normal.

After that, the warning means 13 b issues a warning in accordance withthe speed of approach by using the selected image group (S55). Forexample, when the speed of approach is equal to or higher than areference value, the warning means 13 b issues a higher level of warningthan when the speed of approach is lower than the reference value. Theissuance of a higher level of warning is described hereinafter. Forexample, the warning means 13 b may modify an image belonging to theselected image group so as to highlight the object included in the imageand then output the image to the display device 21. Note thathighlighting may include drawing a rectangle line around an object,placing a mark on or above an object, changing the color of an object orthe like, for example. In other words, the warning means 13 b mayperform processing corresponding to the second warning processingdescribed above.

FIG. 16 is a view showing an example of displaying an image with theposition of an object highlighted in the example 1 of the fourthembodiment. A highlighted display area 43 is displayed on a displayscreen 40 in this example. Note that processing of issuing a higherlevel of warning is not limited thereto, and the warning means 13 b mayoutput a warning tone through the speaker 22, for example.

On the other hand, the case where the level of warning is not highcompared with the above (for example, when the speed of approach islower than a reference value) is also described as an example. In thiscase, the image that belongs to the selected image group is output tothe display device 21 and displayed on the screen. At this time, thewarning means 13 b performs processing such as displaying the imagebelonging to the selected image group on the display device 21 withoutany modification, or, when modifying the image and displaying it on thedisplay device 21, performs processing such as drawing a thinner linearound an object, placing a different mark above an object, or changingthe color of an object to be modified compared with the case where thelevel of warning is high. Further, in the case where the warning means13 b outputs a warning tone through the speaker 22, it performsprocessing such as changing the volume or changing the warning tonecompared with the case where the level of warning is high. In otherwords, the warning means 13 b may perform processing corresponding tothe first warning processing described above.

As described above, in the example 1 of the fourth embodiment, thedegree of movement of an object is determined by using a group of imagestaken by a more appropriate filter in the outside environment during acertain period of time, and it is thereby possible to issue anappropriate warning with use of an image with higher detection accuracy.

Because the object detection accuracy is high in each image that belongsto a group of images taken by an appropriate filter, a difference in theposition of an object between images that are adjacent in chronologicalorder is clear. Thus, the movement of the object can be clearly grasped.On the other hand, because the object detection accuracy is low in eachimage that belongs to a group of images taken by an inappropriatefilter, a difference in the position of an object between images thatare adjacent in chronological order is not clear. Thus, it is difficultto determine whether the object is moving or not. Therefore, the speedof approach is calculated to be relatively higher in the former than inthe latter. Accordingly, the detection accuracy by a filter isrelatively high in the image group where the speed of approach iscalculated to be higher. By outputting an image in the image group withhigh detection accuracy on the screen or the like, a driver canadequately recognize the danger.

Further, for example, when the speed of approach exceeds a referencevalue, it is highly urgent and therefore the level of warning is furtherincreased, thereby giving an appropriate warning to a driver. On thecontrary, when an object is detected in an image taken at a certain timeand no object is detected in an image taken at a later time, it islikely that the object is detected by mistake, or the object has movedaway from the movable body 1 b. In such a case, it is determined not toissue a warning, thereby avoiding an excessive warning.

Example 2

In the example 2, an image with high detection accuracy is selected foreach time period in order to deal with a sudden change in the outsideenvironment. Processing of the example 2 is described hereinafter withreference to FIG. 17, and with reference also to FIG. 18 according toneed. FIG. 17 is a flowchart illustrating the flow of a detailed processin the example 2 of the fourth embodiment. FIG. 18 is a viewillustrating the concept of processing in the example 2 of the fourthembodiment.

First, the determination means 17 compares images corresponding to thetime t1 and selects one image with higher detection accuracy (S61). Forexample, a comparison of detection accuracy between images correspondingto the time may be made by detecting the degree of matching by patternmatching techniques or the like with use of a parameter related to thedetected appearance and a comparative parameter, which are used fordetection results described above, and comparing the results. Next, thedetermination means 17 compares images corresponding to the time t2 andselects one image with higher detection accuracy (S62). Likewise, thedetermination means 17 compares images corresponding to the time t3 andselects one image with higher detection accuracy (S63). Note that theorder of performing Steps S61 to S63 is not limited thereto.

In the example of FIG. 18, the determination means 17 compares detectionresults between the taken image 311 and the taken image 321corresponding to the time t1 and selects the taken image 311 with higherdetection accuracy. Likewise, the determination means 17 comparesdetection results between the taken image 312 and the taken image 322corresponding to the time t2 and selects the taken image 322 with higherdetection accuracy. Further, the determination means 17 comparesdetection results between the taken image 313 and the taken image 323corresponding to the time t3 and selects the taken image 313 with higherdetection accuracy.

After that, the determination means 17 compares the selected images atthe time t1 to t3 in chronological order and calculates the speed ofapproach (S64). In the example of FIG. 18, the determination means 17calculates the speed of approach from the taken image 311, the takenimage 322 and the taken image 313 as one image group.

Then, the determination means 17 determines the degree of movement ofthe object by using the calculated speed of approach (S65). The warningmeans 13 b then issues a warning in accordance with the speed ofapproach by using each of the selected images (S66). Note that, in StepsS65 and S66, the level of warning may be increased as shown in FIG. 16when the speed of approach is equal to or higher than a reference value.

As described above, in the example 2 of the fourth embodiment, it ispossible to appropriately determine the degree of movement of an objectand issue an appropriate warning in the case where there is a suddenchange in the outside environment within a certain period of time. Thecase where there is a sudden change in the outside environment within acertain period of time is, for example, the case where the brightness ofthe surrounding area abruptly changes, such as before and after enteringa tunnel, before and after coming out of a tunnel, when the weatherchanges quickly due to rapid movement of clouds or the like. In such acase, while the detection accuracy of the first image group taken by thefirst filter characteristics is higher until a certain point of time,the detection accuracy of the first image group decreases halfwaythrough, and the detection accuracy of the second image group taken bythe second filter characteristics can become higher. Further, when thebrightness is not stable due to the sunset and clouds, the image groupwith higher detection accuracy can be in alternation as shown in FIG.18.

Because the example 2 makes a comparison of the detection accuracy foreach time an image is taken, it is possible to select images taken byappropriate filter characteristics for the brightness at each time andform an image group with high detection accuracy by a set of theselected images. It is thereby possible to calculate the speed ofapproach with high reliability and issue an appropriate warning.

Example 3

In the example 3, when it is difficult to determine appropriate filtercharacteristics for the outside environment among a plurality of filtercharacteristics, images taken by the respective filter characteristicsare synthesized to enhance the detection accuracy. Processing of theexample 3 is described hereinafter with reference to FIG. 19, and withreference also to FIG. 20 according to need. FIG. 19 is a flowchartillustrating the flow of a detailed process in the example 3 of thefourth embodiment. FIG. 20 is a view illustrating the concept ofprocessing in the example 3 of the fourth embodiment.

First, the determination means 17 compares images corresponding to thetime t1 and specifies the position of an object in the image (S71).Next, the determination means 17 compares images corresponding to thetime t2 and specifies the position of an object in the image (S72).Likewise, the determination means 17 compares images corresponding tothe time t3 and specifies the position of an object in the image (S73).Note that, the processing of specifying the position may calculate theaverage of the positions of an object in the images to be compared.Further, the images to be compared may be synthesized. Note that theorder of performing Steps S71 to S73 is not limited thereto.

In the example of FIG. 20, the determination means 17 specifies, fromthe taken image 311 and the taken image 321 corresponding to the timet1, the position of an object as an intermediate position between them.Likewise, the determination means 17 compares the taken image 312 andthe taken image 322 corresponding to the time t2, compares the takenimage 313 and the taken image 323 corresponding to the time t3, andspecifies the positions of an object as intermediate positions betweenthem.

After that, the determination means 17 compares the specified positionsin chronological order and calculates the speed of approach (S74). Inthe example of FIG. 20, the determination means 17 calculates the speedof approach by using the images 331, 332 and 333 as one image group.

Then, the determination means 17 determines the degree of movement ofthe object by using the calculated speed of approach (S75). The warningmeans 13 b then issues a warning in accordance with the speed ofapproach by using the specified position of the object in each timeperiod (S76). Note that, in Steps S75 and S76, the level of warning maybe increased as shown in FIG. 16 when the speed of approach is equal toor higher than a reference value.

As described above, in the example 3 of the fourth embodiment, theaverage or the like is calculated from detection results of therespective image groups, and therefore the effect of false determinationcan be suppressed. Further, when it is in the early evening, forexample, the brightness in the area around the movable body 1 b isbetween the daytime and the nighttime, and it is sometimes difficult todetermine which of the first filter characteristics and the secondfilter characteristics is appropriate for use. Even in such a case, bycalculating an intermediate position between the two detection results,it is possible to specify the position of an object with adequateaccuracy. It is thereby possible to maintain the object detectionaccuracy by using a plurality of image groups in a comprehensive manner.

Fifth Embodiment of the Invention

A fifth embodiment is a modified example of the fourth embodimentdescribed above. While the image acquisition means 11 b includes twoimaging devices in the fourth embodiment described above, an imageacquisition means according to the fifth embodiment includes one imagingdevice. Note that the configuration of the image acquisition meansaccording to the fifth embodiment is the same as shown in FIG. 10described above, and the illustration and the detailed descriptionthereof are omitted. Note that the switch means 1133 in the imageacquisition means according to the fifth embodiment takes images byswitching a plurality of filter characteristics on a regular basis andcan thereby take images of the same target substantially in parallel bythe plurality of imaging devices.

As described above, the same advantages as in the fourth embodimentdescribed above can be obtained also in the fifth embodiment. Further,any of the examples 1 to 3 described above can be applied to the fifthembodiment.

Note that the detection means 12 b may compare the detection resultsbetween images in each of image groups in chronological order, calculatethe speed of approach to the object for each of the image groups, selectthe image group with the highest speed of approach among the pluralityof image groups, and determine the degree of movement of the object inthe selected image group.

Other Embodiment of the Invention

The above-described examples can be combined as desirable. Specifically,the second embodiment and the third embodiment may be combined.

Although the exemplary embodiment of the present invention is describedin the foregoing, the present invention is not restricted to theabove-described configuration, and various changes, modifications andcombinations as would be obvious to one skilled in the art may be madewithout departing from the scope of the invention.

Arbitrary processing of the on-vehicle device described above may beimplemented by causing a CPU (Central Processing Unit) to execute acomputer program to perform given processing. In this case, the computerprogram can be stored and provided to the computer using any type ofnon-transitory computer readable medium. The non-transitory computerreadable medium includes any type of tangible storage medium. Examplesof the non-transitory computer readable medium include magnetic storagemedia (such as floppy disks, magnetic tapes, hard disk drives, etc.),optical magnetic storage media (e.g. magneto-optical disks), CD-ROM(Read Only Memory), CD-R, CD-R/W, and semiconductor memories (such asmask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM(Random Access Memory), etc.). The program may be provided to a computerusing any type of transitory computer readable medium. Examples of thetransitory computer readable medium include electric signals, opticalsignals, and electromagnetic waves. The transitory computer readablemedium can provide the program to a computer via a wired communicationline such as an electric wire or optical fiber or a wirelesscommunication line.

Further, in addition to the case where the functions of theabove-described exemplary embodiment are implemented by causing acomputer to execute a program for implementing the functions of theabove-described exemplary embodiment, the case where the functions ofthe above-described exemplary embodiment are implemented by this programin cooperation with the an OS (Operating System) or application softwarerunning on the computer is also included in the exemplary embodiment ofthe present invention. Further, the case where all or part of theprocesses of this program are executed by a function enhancement boardinserted into the computer or a function enhancement unit connected tothe computer to implement the functions of the above-described exemplaryembodiment is also included in the exemplary embodiment of the presentinvention.

According to the embodiment, it is possible to provide a warning device,a warning method and a warning program for issuing an appropriatewarning based on detection of an object even under the circumstanceswhere it is difficult to determine the outside environment of a movablebody.

Further, according to the embodiment, it is possible to provide awarning device, a warning method and a warning program for maintainingthe accuracy of detecting the movement of an object and issuing anappropriate warning even under the circumstances where it is difficultto determine the outside environment of a movable body.

The exemplary embodiment is applicable to a warning device that ismounted on a movable body including a vehicle, and it has an industrialapplicability.

What is claimed is:
 1. A warning device comprising: an image acquisitionunit configured to acquire a plurality of images respectively based on aplurality of filter characteristics; a detection unit configured toperform detection of a specified object on each of the plurality ofacquired images; and a warning unit configured to issue a specificwarning when the object is detected from at least any one of theplurality of acquired images, wherein the warning unit issues a higherlevel of warning when the object is detected from all of the pluralityof images than when the object is detected from some of the plurality ofimages.
 2. The warning device according to claim 1, wherein when theobject is detected from two or more of the plurality of images, thewarning unit issues a specific warning by using the image in which theobject is detected with higher accuracy.
 3. The warning device accordingto claim 2, wherein the detection unit calculates an index valueindicating a level of detection of the object on each of the pluralityof acquired images, and the warning unit determines an image in whichthe index value exceeds a specified value as the image in which theobject is detected, and when the object is detected from two or more ofthe plurality of images, the warning unit issues a specific warning byusing the image in which the index value is highest.
 4. The warningdevice according to claim 1, further comprising: a setting unitconfigured to select one of a first mode where the image acquisitionunit takes an image by one filter characteristics and a second modewhere the image acquisition unit takes an image by a plurality of filtercharacteristics in accordance with specified conditions, and sets theselected mode to the image acquisition unit.
 5. The warning deviceaccording to claim 4, further comprising: a position informationacquisition unit configured to acquire current position information ofthe warning device, wherein when the acquired position information iswithin a range of a specific area on map information, the setting unitselects and sets the second mode to the image acquisition unit.
 6. Thewarning device according to claim 1, wherein when the object is detectedby the detection unit, the warning unit issues a specific warning bydisplaying the detected image with a position of the object highlightedon a screen.
 7. The warning device according to claim 1, wherein theimage acquisition unit includes a plurality of imaging devicesrespectively corresponding to the plurality of filter characteristics,and each of the plurality of imaging devices takes an image bycorresponding filter characteristics.
 8. The warning device according toclaim 1, wherein the image acquisition unit includes one imaging device,and the imaging device takes an image by switching the plurality offilter characteristics on a regular basis.
 9. A warning methodcomprising: an image acquisition step of acquiring a plurality of imagesrespectively based on a plurality of filter characteristics; a detectionstep of performing detection of a specified object on each of theplurality of acquired images; and a warning step of issuing a specificwarning when the object is detected from at least any one of theplurality of acquired images, wherein the warning step issues a higherlevel of warning when the object is detected from all of the pluralityof images than when the object is detected from some of the plurality ofimages.
 10. A non-transitory computer readable medium storing a warningprogram for causing a computer to execute: an image acquisition step ofacquiring a plurality of images respectively based on a plurality offilter characteristics; a detection step of performing detection of aspecified object on each of the plurality of acquired images; and awarning step of issuing a specific warning when the object is detectedfrom at least any one of the plurality of acquired images, wherein thewarning step issues a higher level of warning when the object isdetected from all of the plurality of images than when the object isdetected from some of the plurality of images.